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Statistical Significance Versus Clinical 
Relevance in Cardiovascular Medicine 



Ronnie Willenheimer 



Evidence-based medicine is the foundation of ev- 
eryday clinical practices and large clinical trials 
investigating the effects of various interventions on 
morbidity and survival and generally provide the 
most robust evidence. Cardiovascular medicine is 
considered one of the most evidence-based disci- 
plines of medicine. However, there are a number of 
limitations to the general applicability of clinical trial 
results in cardiovascular medicine. Although gen- 
erally useful to the clinician, clinical trials have often 
been suboptimally designed from 1 or several 
points of view. As a consequence of flaws in the 
design and the execution of the trials, statistical 
significance is quite often not equal to clinical rel- 
evance. This article outlines some of the shortcom- 
ings of designing and carrying out clinical trials, as 
well as inadequacies concerning the publication, 
interpretation, and implementation of the trial re- 
sults. Evidence-based medicine is obviously not 
always as solid as one might think. 
Copyright © 2001 by W.B. Saunders Company 



Evidence-based medicine has, during the past 
years, emerged as a key feature of everyday 
clinical practice. The findings of large clinical tri- 
als, designed to investigate the effects of an inter- 
vention on morbidity and morialily, conslilute 
the backbone of evidence- based medicine. Car- 
diovascular medicine is a discipline considered 
being evidence-based to an unusually high degree. 
One reason for this is that a multitude of large 
clinical trials during the last 2 decades have doc- 
umented the effects of different pharmacologic 
and nonphannacologic interventions on morbid- 
ity and mortality in various cardiovascular condi- 
tions. Today, we base the treatment of most car- 
diovascular conditions to a large extent on such 
documentation. 

However, this documentation may not always 
be as solid as we like to think. A statistically sig- 



nificant finding of a clinical trial does not neces- 
sarily imply that it is clinically relevant. There are 
a number of factors to bear in mind when making 
a decision about whether a statistically significant 
trial result is of chnical relevance. Among the fac- 
tors to take into account is whether the patient 
population is representative of the clinical situa- 
tion with regard to age, sex, race, concomitant 
diseases, and concomitant medication. If the trial 
population differs from your patients in chnical 
reality in 1 or more of these aspects, the trial result 
may be of limited interest to you. Other important 
aspects to consider are whether the study duration 
is relevant, the endpoints are vaUd, and the differ- 
ence between the intervention groups really is of 
clinical interest. The result of a relatively short- 
term clinical trial may not be relevant to a long- 
term treatment effect, a very common flaw with 
clinical trial evidence. It is important to question 
whether the trial was large enough and whether 
the absolute difference, not just the relative differ- 
ence, among the intervention groups is convinc- 
ing. One should also consider whether hidden 
factors could affect the relevance of the clinical 
trial situation for everyday clinical practice, ie, if 
real-life patients can be expected to have a similar 
benefit of the intervention as the trial patients (eg, 
the side effect profile of a more efficacious drug 
may be such that real-life patients can be expected 
to comply less well with it than the trial patients). 
In such a case, the less efficacious drug in the trial 
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may do more good in real life if it is better toler- 
ated and, therefore, more frequently used and 
complied with. It is also important to take into 
account whether the findings are concurrent with 
the trial aims and hypotheses, or if they are unex- 
pected. Unexpected findings generally have a low 
evidence value and need to be confirmed in other 
trials. This is really independent of how big the 
trial is and how statistically significant the differ- 
ence is. Furtfiermore, a general rule is that the 
findings of one single trial should be supported by 
other trial results. 

Based on large clinical trials, interventions 
commonly considered established with regard to a 
benefit on mortality and morbidity include: 

1. acetylic salicylic acid (ASA) and thrombolytic 
drugs in patients with acute myocardial infarc- 
tion; 

2. angiotensin-converiing enzyme (ACE) inhibi- 
tors in patients with left ventricular systolic 
dysfunction irrespective of symptoms of heart 
failure and in patients with high cardiovascular 
risk but without heart failure or obvious left 
ventricular systolic dysfunction; 

3. j3-receptor blockers and spironolactone on top 
of an ACE inhibitor in patients with chronic 
heart failure; 

4. diuretics, )3-receptor blockers, calcium chan- 
nel blockers, and ACE inhibitors in hyperten- 
sion; and 

5. statins in the secondary prevention of coronary 
artery disease. 

The benefits of other interventions may be con- 
sidered less well proven, such as: 

1. j3-receptor blockers after myocardial infarc- 
tion; 

2. glycoprotein llbAlla-receptor blockers in un- 
stable angina; 

3. calcium channel blockers in several groups of 
patients with cardiovascular disease; 

4. digoxin in heart failure with and without atrial 
fibrillation; 

5. ASA administration after the acute phase of 
myocardial infarction; 

6. class 1 and 111 antiarrhythmic drugs in post- 
myocardial infarction patients; 

7. angiotensin rype-1 receptor blockers in heart 
failure; 



8. aggressive and invasive treatment of unstable 
angina in general; and 

9- implanted cardiac defibrillators for severe ar- 
rhythmia. 

However, both the negative and the positive 
interpretations of the clinical trial results may be 
challenged from several points of view. In this 
article, controversial generic factors regarding 
clinical trials and the basis they provide for the 
treatment of cardiovascular diseases is discussed. 
Detailed analyses of specific inter\^entions is dis- 
cussed by other contributors to this symposium. 

Reasons for a Lack of Conclusiveness 
of Cardiovascular Clinical Trials 

The ultimate purpose of clinical trials is that, as 
clinicians, we should be able to apply the trial 
results to the patients we treat in ever>'day clinical 
practice. This is far from generally the case. In 
many patient groups, the evidence from the clini- 
cal trials is more or less questionable regarding its 
direct applicabOily to nonstudy patients. 

Age 

The mean age of the patients in the major cardio- 
vascular mortality and morbidity trials has often 
been substantially lower than that encountered in 
clinical practice, because elderly patients have of- 
ten been excluded. One common reason for ex- 
cluding older patients is competing comorbidi- 
ties. This has obvious relevance when mortality is 
an endpoint. For example, most of the trials inves- 
tigating the impact on mortality of ACE inhibitor 
intervention in patients with left ventricular sys- 
tolic dysfunction and heart failure have included a 
patient population considerably younger than the 
average heart failure patient.* ^ Consequently, al- 
though some trials investigating the effects of ACE 
inhibitor intervention in cardiac failure after myo- 
cardial infarction have included a population 
somewhat more relevant with regard to age,*^*'' we 
know very little about the effects of ACE inhibi- 
tors in the elderly. Because the elderly patients 
with heart failure constitute the main target pop- 
ulation for ACE inhibitor treatment, obviously 
this patient category should have been more ex- 
tensively represented in the ACE inhibitor trials. 
The same is true for the mortality trials with ^-re- 
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ceptor blockers/-^° spironolactone/^ *^ and digi- 
talis^ ^ in heart failure. 

The situation is largely similar with the major 
intervention trials in acute myocardial infarction 
and stable and unstable angina. With hyperten- 
sion, the problem is less important because there 
are some major intervention trials in hypertension 
that have included a substantial part of elderly 
patients/'*'^' 

Gender 

hi general, the major cardiovascular mortality and 
morbidity trials have included few women. Be- 
cause women constitute half the target population 
in clinical practice, this is a major flaw. Indeed, it 
is difficult to draw safe conclusions regarding a 
treatment effect in women from virtually all car- 
diovascular mortality trials to date. A partial ex- 
planation for the relative paucity of data on 
women is the interrelationship with the problem 
of age, the cardiovascular disease debut is com- 
monly delayed several years in women compared 
with men. It is imperative that future trials include 
a more representative sample of women. Simply 
by hmiting the number of exclusion criteria, such 
as high age, the proportion of women should in- 
crease. However, with consecutive inclusion of 
patients, it is still possible that there rnust be an 
excess number of men to include a relevant num- 
ber of women in a trial. A possible solution to this 
problem is to consecutively include only every 
second or third man, but all women. Of course, 
another alternative would be to conduct separate 
trials in women, but then one would not be able to 
directly compare men and women with regard to 
the treatment effect. 

Extensive Exclusion 

Cardiovascular mortality trials usually have pre- 
sented a long list of exclusion criteria. Besides 
excluding many women and elderly patients, 
these criteria have lead to the exclusion of an ad- 
ditional, substantial part of the patient population 
with the condition at issue. One example is the 
very common exclusion criterion "other serious 
diseases that may affect survival during the study 
period." Another reason for exclusion is when 
comorbidities dictate the choice of drug to be 
used, which has been the case, for example, in 
hypertension trials. In clinical practice, we often 



do not withhold a certain treatment because a 
patient would have met exclusion criteria applied 
in the trials providing the knowledge about the 
effects of the treatment in question. This is obvi- 
ously not evidence-based medicine. Because such 
patient categories have not been investigated, we 
cannot be confident that the results shown for the 
intervention are applicable to these patients. It 
should be possible to use fewer exclusion criteria 
in future trials, to reproduce the clinical reality. 

Selection of Low-Risk Patients 

Virtually all drugs have the potential to cause 
some harm besides the benefit they confer. There 
is little chance of showing benefit of an interven- 
tional drug if the patients investigated in a trial are 
at low risk of suffering the endpoints examined. In 
such trial populations, there is a risk that the 
harmful effects of the drug will prevail. 

The inclusion and exclusion criteria applied in 
clinical trials have sometimes caused a selection of 
a low-risk study population. Examples are the 
CAST trials,!^'^^ and to some extent, the SWORD 
trial. These trials examined the effects of class 
pfi.i7 ^nd class lIPs antiarrhythmic drugs in sur- 
vivors of a myocardial infarction. In these trials, 
the mortality among placebo-treated patients was 
lower than intended and the inter\^ention was 
shown to cause harm. These studies are largely 
responsible for the widespread negative view of 
antiarrhythmic therapy among clinicians. This 
skepticism has commonly been extrapolated also 
to high-risk patients, perhaps with the exception 
of amiodarone. However, it might not be correct 
to apply these trial results to the entire population 
of patients with arrhythmia because we know lit- 
tle about the effects of these drugs in a high-risk 
population. If these drugs have the potential to 
cause harm as well as benefit, it will obviously not 
be possible to show any beneficial effects without 
examining a high-risk population. The DIAMOND 
study, which examined the effect of the class 111 
antiarrhythmic agent dofetilid, included a study 
population with a considerably higher risk than in 
the CAST and SWORD trials. Furthermore, the 
study design aimed to lower the risk of proar- 
rhythmic effects by excluding patients with early 
proarrhythmic reactions. This is obviously a chn- 
ically relevant approach. The study showed a neu- 
tral effect on mortality and a significant reduction 
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of heart failure hospitalization.-''^ It is likely that 
the result would have been different if a low-risk 
population, as in CAST and SWORD, had been 
included. Thus, it is vital not to extrapolate results 
of clinical trials to other patient populations than 
those actually studied. 

Another reason for a lower than intended mor- 
tality in a trial is because power calculations are 
based on old data, not taking into account the 
constantly improving prognosis in many cardio- 
vascular conditions. If there is an actual mortality 
difference between 2 treatment regimens, it is in 
general more difficult to show this in a low-risk 
population than among high-risk patients. In a 
low-risk population, it may simply not be possible 
to lower mortality much further with any inter- 
vention. If mortality in a trial is lower than ex- 
pected, it is possible to reach the prespecified 
number of deaths by increasing the time of follow- 
up. However, all-cause mortality may, with time, 
be increasingly diluted by other reasons than the 
relevant cardiac/cardiovascular disease. If the in- 
vestigated treatment only has an effect on cardiac/ 
cardiovascular mortality, which usually is the 
case, there may be no significant mortality differ- 
ence between the 2 treatment groups. 

The GUSTO IV-ASC^^ and GUSTO V^^ tiials 
may be regarded as recent examples of how a mis- 
calculation of the mortality rate may have affected 
the results. In both trials, the mortality in the 
control patients was substantially lower than ex- 
pected. The former showed no mortality benefit of 
the glycoprotein Hb/llla-receptor blocker abcix- 
imab as add-on therapy to ASA and heparin in 
patients with unstable angina. The GUSTO V trial 
showed a similar mortality effect of a half-dose 
reteplase combined with a full-dose abciximab 
compared with full-dose reteplase in patients with 
acute myocardial infarction. 

Changing ConcomitanL Therapy 

One reason for the constantly improving progno- 
sis in many cardiovascular conditions is improved 
drug therapy. Many therapeutic interventions in 
cardiovascular disease have been documented re- 
garding mortality and morbidity some years ago 
when concomitant therapy was different than it is 
today. For example, over the recent years, there 
has been a continuously increasing use of statins 
in patients with cardiovascular diseases. Conse- 



quently, the effects of an intervention (eg, ASA in 
patients with coronary artery disease) may he dif- 
ferent today when patients more often are on a 
statin compared with how it was some years ago, 
when the trials showing benefit of ASA were per- 
formed. 

The other way around may also be true. Com- 
pared with more recent trials, older studies on 
statins showed considerably higher placebo group 
mortality.^"* This is partly because of selection of 
patients with lower cholesterol values but also indi- 
cates that concomitant drug therapy and invasive 
treatment have changed in a clinically relevant man- 
ner. Thus, it is not obvious that the results of the 
older trials are entirely valid today. A more recendy 
examined compound may be a better choice when 2 
different drugs of the same drug class have shown 
similar results in similar patient populations. 

Another example is 0-blocker treatment in pa- 
tients with acute myocardial infarction and in pa- 
tients after a myocardial infarction. The evidence 
of a beneficial effect on mortality and morbidity of 
)3-blockadc in these patient categories originates 
mainly from studies performed in the pre thrombo- 
lytic era. Thus, we cannot be confident that 
/^-blocker treatment causes the same benefit in pa- 
tients who have received thrombolytic therapy. It is 
possible that the effect is less pronounced because 
the thrombolytic therapy has already reduced the 
risk substantially in these patients. However, it is 
also possible that the effect is greater because, as a 
result of the thrombolytic therapy, some padents 
who would have died withoiU thrcjmbolysLs vnW sur- 
vive to become high-risk patients. Of course, differ- 
ences in statin use may also determine the effects of 
/3-blockade. However, the recently performed CAP^ 
RICORN trial shows diat, in patients with left ven- 
tricular systolic dysfunction after a myocardial in- 
farction, carvedilol bestows a benefit in terms of 
mortality and niorbidity-^"^ 

Inadequate Study Duration 

We usually extrapolate a treatment effect docu- 
mented during a limited time of follow-up to a 
longer period of time, perhaps life long. For how 
long should j3-blockcr therapy after myocardial 
infarction be administered? We tend to consider 
this a hfe-long treatment. However, with the ex- 
ception of a 6-ycar open follow-up of the Norwe- 
gian Timolol Study, no trial showing benefit 
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from ^-blockade in postinfarction patients has 
had a longer study duration than 3 years. Thus, 
there is little evidence of a life-long beneficial ef- 
fect of this therapy in these patients. Furthermore, 
the importance of safety is amplified when long- 
term effects are considered. 

How should one handle a patient who develops 
symptomatic bradycardia and who is treated with 
a j3-blocker for 5 years after a myocardial infarc- 
tion? Should the /3-blocker be permanently dis- 
continued if the bradycardia resolves when the 
^-blocker is interrupted? Or, should the patient 
receive a pacemaker to allow for continuing 

blocker treatment? 

The same question could be asked in the case of a 
similar scenario in a patient with congestive heart 
failure. All of the large mortahty trials examining the 
effects of a ^blocker in patients with heart failure 
have been rather short. '■'^'i*^ -'^ The question would 
also be relevant in patients with hypertension. 

Inadequate Testing of The Hypothesis 

It goes \\^thout saying that a trial must have a design 
that allows for adequate testing of the hypotheses. 
However, this is generally far from the case. Even 
some of the most important trials have important 
design flaws. An example of a megatrial with a de- 
sign that did not allow for conclusive testing of a 
hypothesis is the ISlS-4 trial. The interpretation of 
the study result was that intravenous magnesium 
given to patients with acute myocardial infarction 
had no beneficial effect. This result was in contrast to 
an earlier smaller study^® showing that magnesium 
significantly reduced mortality in patients with 
acute myocardial infarction. Because ISIS-4 was a 
much larger study clinicians ceased (or never 
started) using magnesium on this indication* How- 
ever, it has been argued that magnesium can only be 
of benefit if given before thrombolysis,^^ which 
was not the case in lSlS-4.^'^ Thus, contrary to the 
general belief, a beneficial effect of magnesium in the 
acute myocardial infarction situation was not pre- 
cluded by lSlS-4. A new study with a proper design 
will investigate this hypothesis.-^" 

The Problem of Implementing Trial 
Results Into Clinical Practice 

There is usually a long delay from the lime of 
proven benefit of an intervention in clinical trials 



and a subsequent change of clinical practice. In 
some cases, it seems almost impossible to imple- 
ment the trial results despite seemingly well-doc- 
umented beneficial effects of an intervention. One 
example is the use of ACE inhibitors in heart fail- 
ure, which has been reported to be only around 
30% in most western societies despite docu- 
mented beneficial effects on mortality, morbidity, 
and physical capacity in several trials. Fur- 
thermore, if prescribed at all in cHnical practice, 
ACE inhibitors are frequently used in lower doses 
than in clinical trials. This is alarming because the 
effects on mortality and morbidity have been 
proven only for target doses. Indeed, it has been 
shown that the benefit of a lower dose of an ACE 
inhibitor is less pronounced compared with a 
higher dose.^^ 

According to a British investigation, a factor 
strongly contributing to the low prescription of 
ACE inhibitors to patients v^th heart failure is a 
fear among physicians of possible side effects, es- 
pecially hypotension and renal failure. This fear 
is in contrast to the fact that the major studies on 
ACE inhibitors in heart failure have shown these 
agents to be safe, with a low incidence of these side 
effects. A possible explanation is that patients in 
clinical practice often differ importantly from 
those included in the trials. The most obvious 
difference between clinical practice and clinical 
trials is that patients in clinical practice are usually 
substantially older than those in the trials. Often 
these patients also have important concomitant 
diseases, which generally were not present in pa- 
tients in the clinical trials. Consequently, clini- 
cians do not feel convinced that the results of the 
chnical trials are applicable to their daily practice 
patients with heart failure and might fear that the 
side-effect profde is substantially more unfavor- 
able compared with what has been shown in the 
trials. This could of course lead to a reluctance to 
prescribe ACE inhibitors in elderly patients with 
multiple diseases. To avoid problems of this na- 
ture in the future, it is imperative to perform the 
trials in patients such as those generally found 
with the disease in question. Perhaps the Heart 
Outcomes Prevention Evaluation (HOPE) study 
may help increase the use of ACE inhibitors since 
it showed clear benefit of ACE inhibition in pa- 
tients whom we commonly meet in everyday clin- 
ical practice.^"* 
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In this conLext, it is of course also important 
that the study results are convincing in terms of 
absolute differences in mortality and morbidity. 
One may question whether the result of the large 
mortality trials examining the effects of ACE in- 
hibitors in heart failure really is that impressive, 
especially in view of the fear of side-effects, which 
is obviously shared by many physicians. The Studies 
of Left Ventricular I>ysfunction treatment trial, for 
example, shows that the chance of surviving for the 
next 3.5 years without an ACE inhibitor is 60.3%, 
whereas it is 64.8% with an ACE inhibitor.^ This 
difference might not be convincing enough to the 
clinician, especially not if one fears that the result 
might be even less pronounced in the elderly pa- 
tient with recently diagnosed heart failure in 
whom one is about to start treatment. A clear 
result from an elderly patient cohort would obvi- 
ously be of much help in this situation. 

There are numerous examples of clinical trials 
in cardiovascular disease showing impressive rel- 
ative reductions of mortality and morbidity with- 
out showing much increase in the chance of sur- 
vival or event-free survival in absolute terms. This 
may be owing to a relatively low risk in the popu- 
lation examined or to a short study duration. It is, 
of course, survival or event- free survival in abso- 
lute terms that is important to the physician and 
patient when making a decision about therapy. 
Consequently, it is imperative to express the study 
results in absolute terms and to design the study to 
get the best possible results in absolute terms. The 
importance of an adequate study duration is dis- 
cussed further below. 

A consistency of results in different trials of the 
same agents or the same class of agents should 
help to convince the clinicians and facilitate the 
implementation of a certain therapy. Thus, it is of 
importance to verify the results of any cHnical trial 
in other independent trials. 

Should Clinical Trials Be Stopped 
Early Because of a Statistically 
Significant Beneficial Effect of 
the New Intervention? 

The ethics of clinical trials is sometimes compli- 
cated. The ethical considerations of the study pa- 
tients may be in opposition to what is best for the 
general population with the condition at task. It 
goes without sapng that, in the best interest of 



patients in the intervention group, studies show- 
ing a negative effect of an intervention must be 
stopped as soon as the negative effect is evident. 
But does this necessarily mean that chnical trials 
should be stopped early because of a statistically 
significant beneficial effect of the new inter\-en- 
tion? 

Early termination of a chnical trial because of 
the benefii of the new intervention should, of 
course, always be based on clearly defined, pre- 
specified stopping rules. However, it may be ar- 
gued that this practice is unscientific and causes 
more harm than benefit. The reason for early ter- 
mination of trials showing benefit is that the pa- 
tients in the control group must be given the op- 
portunity to receive the new, better treatment as 
soon as possible. However, the overall purpose of 
a chnical trial is that, if proven superior, the new 
treatment should be given as soon as possible to 
all patients like those in the trial. This concern 
about the control group may be in conflict with 
the best interest of the general population with the 
condition studied. In fact, it may not even be in 
the best interest of the control group. 

Small Absolute Difference in the Treatment 
Effect Because of Early Termination 

The number of deaths differing between 2 study 
groups may be quite small, although there is a 
highly statistically significant beneficial effect of 
the intervention. In the United States carvedilol 
trials, 1,094 patients with heart failure were ran- 
domized to double-blind treatment with placebo 
(n = 398) or carvedilol (n 696) on top of stan- 
dard therapy, including an ACE inhibitor.^ After a 
mean follow-up of just over 6 months, the study 
was prematurely terminated owing to a highly sta- 
tistically significant (P < .001) 65% relative mor- 
tahty reduction in the carvedilol group. The num- 
ber of deaths was 22 (3.2%) in the carv^edilol 
group versus 31 (7.8%) in the placebo group, ie, a 
difference of 9 deaths. Had the groups been 
equally large, the difference would have been 25 
deaths (given a similar mortality rate). The clini- 
cian might question whether a result like that 
should change the therapeutic approach in heart 
failure. Indeed, many cUnicians were not con- 
vinced by that result. Consequently, most patients 
with heart failure did not receive this new treat- 
ment. Thus, the result of the early termination of 
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Fig 1 . Different hypothetic mortality curves In clinical trials. The vertical lines indicate the respective time point of 
the earty trial termination. The curves end at the predetermined time of follow-up. (Abbreviations: a, active pe, the 
new therapy]; c, control pe, the old standard therapy].) (Reprinted with permission from the BMJ Publising Group. 
Willenhelmer R, Dahlof B, Gordon A: Clinical trials In cardiovascular medicine: Are we looking for statistical 
significance or clinical relevance? Heart 84:129-133, 2000.) 



the study was that, although the relatively small 
group of patients in the study control group (per- 
haps) received the new therapy, the vast majority 
of the patients with heart failure in the world did 
not. If the new therapy is truly effective, this is 
certainly an example of how concern for the study 
patients can be a disadvantage to the general pop- 
ulation with the condition studied. Had the study 
duration been longer, the effect would have been 
more evident, provided that the mortality curves 
had been as in Fig lA. In that case, it is likely that 
clinicians in general would have been more con- 
vinced of a beneficial effect. Consequently, the 
vast majority of patients with heart failure would 
have been given the benefit of the new treatment, 
whereas the control patients of the study would 
have had to wait a little longer to receive the new 
treatment. It is surely worth discussing which ap- 
proach is best from an ethical point of view. 

Irrelevant Time of Follow-Up Because 
of Early Tenninatiojx 

An early termination of a study may lead to a 
different mean time of follow-up than the one 
intended. Strictly scientifically, the hypothesis 
may not have been properly examined in such a 
case. The shorter time of follow-up may also be 
biologically questionable. How valid is a 6-month 
or 1-year mortality difference in a heart failure 
population with an expected mean sur\'ival time 
of around 6 years? Do we know if the mortality 
cun^es would have been of type A, B, or C (Fig 1), 



if the study had gone on for longer time? It is 
obviously important to determine a biologically 
valid time of follow-up and design the study ac- 
cordingly. 

Greater Impact oJProfessional Study Patients 
Because of Early Termination 

In many studies of chronic conditions, a consid- 
erable part of the study population are profes- 
sional study patients. These are survivors who are 
able to participate in many clinical trials simply 
because they survive long enough. They may also 
be exceptionally compliant to treatment. Conse- 
quently, they constitute a selected group of pa- 
tients who are not necessarily representative of 
the general population with the chronic condition 
studied. The inclusion of these patients in clinical 
trials always makes it questionable to extrapolate 
results from clinical trials to the general popula- 
tion with the condition. These professional study 
patients are often included in the beginning of the 
study period because they are available, whereas 
ordinar)^ patients usually are included later. The 
professional study patients will therefore contrib- 
ute the most to the total time of follow-up. The 
relative contribution of these patients to the time 
of follow-up is greater the shorter the duration of 
the study. Consequently, if a trial is stopped early, 
it may be more questionable to extrapolate the 
result because of the inclusion of professional 
study patients than if the study had gone oh for 
longer time. 
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Early Termination May Not Baiefit the 
Control Patients 

It is not obvious that control patients benefit from 
premature termination of a trial showing benefit 
of a new intervention. If the benefit of the inter- 
vention is mostly or only because of an early ef- 
fect, the control patients may not derive much 
benefit when given the new treatment later, for 
example, as in postmyocardial infarction trials. 

Furthermore, this early treatment effect may 
not be detected if the study duration is short. In 
that case, it may look as if the effect of the inter- 
vention is as in Fig 1 A (ie, a continuous separation 
of the curves). However, a longer study duration 
would show thai there is only separation of the 
curves in the early phase of the intervention, after 
which the curves are parallel. This information is 
obviously important to obtain because it teaches 
us to start treatment early, which is of benefit to 
future patients in clinical practice. This is another 
reason for not stopping trials earlier than intended 
because of a beneficial effect of the intervention. 

The Importance of the Order in 
Which Classes of Drugs Arc 
Investigated 

The order in which drug classes are investigated 
with regard to mortality and morbidity has con- 
siderable impact on therapeutic recommenda- 
tions. This is not a particularly scientific approach 
to therapy and is an example of how evidence- 
based medicine may be potentially confounding. 
For example, the effects on mortality and morbid- 
ity in patients with chronic heart failure were ex- 
amined and established for ACE inhibitors before 
jS-blockers. Consequently, guidelines generally 
recommend that ACE inhibitors be initiated be- 



fore ^-blockers. The impact on mortality of ACE 
inhibitors is approximately 20% to 25% in com- 
parison with placebo, on top of diuretics and, to 
some extent, digitahs.^^ addition of a jS-blocker 
on top of standard treatment for heart failure, in- 
cluding an ACE inhibitor, leads to a further mor- 
taUty reduction of approximately 35%.^'^°-^^ 
Therefore, it is quite possible that monotherapy 
with a ^-blocker would be more efficacious than is 
monotherapy with an ACE inhibitor. 

Many patients cannot tolerate target doses of 
both an ACE inhibitor and a ^-blocker. Because 
therapy according to guidelines is usually started 
with an ACE inhibitor, many patients will be ti- 
trated to the target dose of the ACE inhibitor but, 
because of intolerance, receive suboptimal dos- 
ages of the j3-blocker. Given the possibility that 
)3-blockers might be more efficacious than ACE 
inhibitors, in many individuals this could result in 
a lesser treatment effect compared with if treat- 
ment was started with a ^-blocker. It is important 
to investigate in which order therapeutic regimens 
should be initiated. Therefore, it is imperative that 
the rules of ethics for clinical trials allow lor such 
trials to be conducted. 

This problem will likely increase with the num- 
ber of compulsory agents for the treatment of a 
condition, such as chronic heart failure. Who 
knows, the fifth class of agents to be tested might 
be the most efficacious one, but we will never be 
able to find that out if we can only investigate the 
efficacy of this agent on top of all the other 4 
agents. 

A study design that would at least partly offer a 
solution to this problem is shown in Table L By 
using a study design like this, the new treatment 
regimen can be tested, not only as an add-on to 
established treatment, but also instead of each of 



Table 1. A Clinical Trial Design 



Study Group 






Regimen 






1 


A 


B 


C 


D 


E 


2 


A 


B 


C 


D 


Placebo 


3 


A 


B 


C 


Placebo 


E 


4 


A 


B 


Placebo 


D 


E 


5 


A 


PlacetK> 


C 


D 


E 


6 


Placebo 


B 


C 


D 


E 



NOTE The clinical trial design allows for assessment of the effects of a new treatment regimen (E) on top of, as well as 
instead of, established treatment regimens (A-0). Several classes of agents comprise the standard treatment. 
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the established therapies. Thus, it would allow for 
assessment of the best combination of drugs. This 
design for future trials renders a way to avoid 
potentially disadvantageous polypharmacy, which 
could be associated with unforeseen interactions 
and side effects that could endanger the patient*s 
health.^f' 

The Extrapolation of Trial Results 

Quite often we have no evidence at all to rely on 
when deciding how to treat our patient. Obvi- 
ously, we have to treat the patient as best as pos- 
sible based on the facts available. In such cases, we 
often extrapolate results from clinical trials per- 
formed in patients resembling the patient at hand 
as much as possible. This is a pragmatic way of 
handling the problem, and it may be considered a 
reasonable approach. 

A similar situation arises when a new treatment 
regimen has been shown to be very effective in a 
particular group of patients. Should we then ex- 
trapolate these data to similar patient populations 
and not wait for evidence also in these patient 
populations? Strictly scientifically speaking, the 
answer would be no. However, this might not be 
in the best interest of the patients. 

For example, ACE inhibitors were first proven 
beneficial with regard to mortahty and morbidity 
in patients with chronic symptomatic heart fail- 
ure. Later, they were proven similarly effica- 
cious in patients with asymptomatic left ventricu- 
lar systolic dysfunction^ '* *^ as well as in patients 
with often transient signs of heart failure after an 
acute myocardial infarction. ^ Finally, they were 
proven efficacious in patients with established 
atherosclerotic disease without heart failure or left 
ventricular systolic dysfunction.^'* 

The common denominator of all these condi- 
tions is a similar pathophysiology with neurohor- 
monal activation and a continuous remodeling 
process, probably explaining the similar benefit of 
ACE inhibition. ^7 Today the idea of a cardiovas- 
cular continuum has been widely accepted (Fig 
2). Based on this, one might argue that a treatment 
proven efficacious in 1 or several conditions 
within the cardiovascular continuum would also 
be beneficial for other manifestations of the com- 
mon disease, at least if the intervention affects any 
common denominator. 
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Fig 2. The cardiovascular continuum. Various cardio- 
vascular diseases are linked and may be regarded as 
different manifestations of a common basic disorder. 
Common denominators are neurohormonal activation 
and remodeling of ttie myocardium, the blood vessels, 
the renal mesangtum, and so on. Neurohormonal ac- 
tivation and mechanical overload are the main factors 
behind remodeling and the progression of the cardio- 
vascular disorder. 



Consequently, if one were to apply this theory 
in clinical practice, it would not be necessar>' to 
await definite results from large clinical trials be- 
fore using a novel treatment in a particular dis- 
ease, if the treatment was already proven benefi- 
cial in a related condition. Had this approach been 
used with regard to ACE inhibitors, many patients 
with atherosclerotic disease without heart failure 
or left ventricular systolic dysfunction would have 
had a longer and belter life several years ago. How- 
ever, this is easily said in hindsight and might not 
always be true. 

Sometimes evidence of benefit of a drug in a 
particular category of patients (A) has been ex- 
trapolated to another, similar patient category (B) 
because there has been no evidence to rely on for 
B. In such a case, it seems reasonable to change 
therapy if a new drug is proven efficacious in pa- 
tient category B. However, in clinical practice this 
is not always done. Many clinicians tend to use 
only 1 agent of a class in most or all conditions and 
in most or all patient categories, in which this 
class of agents has been documented. However, 
other drugs in the class may be those with proven 
benefit in many of these conditions and patient 
categories. Fxamples of classes of drugs subject to 
this behavior iri clinical practice are ACE in- 
hibitors, j3-blockers, and statins. Because a possi- 
ble difference between various ACE inhibitors, 
/^-blockers, and statins is not precluded, this is not 
good evidence-based medicine. 
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Another example of questionable evidence- 
based medicine is when a particular drug of a class 
of drugs is very commonly used, although the 
evidence of benefit is ahnost entirely shown for 
other drugs in the class. An example of this is the 
widespread use of atorvastatin despite very little 
evidence of benefit with regard to morulity and 
morbidity. Because there are other drugs in the 
class that are very well documented, this seems 
unnecessary.23 The reason in this particular case 
may be that physicians confuse efficacy with re- 
gard to the surrogate endpoint cholesterol and 
low-density lipoprotein cholesterol with an effect 
on mortality and morbidity. It may be true that 
these 2 entities are strongly interrelated, but this 
has not been proven. This brings us to the next 
question. 

Can We Accept Surrogate Endpoints 
for Mortality? 

A possible alternative to extrapolation of trial re- 
sults to similar patient populations, which would 
also speed up the introduction of new efficacious 
therapies in cardiovascular medicine, is to use sur- 
rogate endpoints for mortality in clinical trials. 
Clinical trials using mortality as the primary end- 
point take a long time to conduct and demand 
huge resources. This obviously limits the number 
of conditions and populations in which new treat- 
ment regimens can be investigated. If there was a 
good surrogate for mortality, this would clearly 
increase the capacity for interventions in various 
diseases and subpopulations with the disease at 
hand. At present, there is no such widely accepted 
surrogate. 

It is well knovi^n today that exercise capacity, 
left ventricular ejection fraction, and signs and 
symptoms are useless as surrogates for mortality 
and even morbidity in patients with chronic heart 
failure. Interventions improving these parameters 
may very well have a negative effect on survival 
(eg, as seen with most positive inotropic agents). 
Similarly, blood pressure cannot he used as a sur- 
rogate for mortality in hypertension. However, 
B-natriuretic peptide (BNP) has been brought 
forward as a possible surrogate for mortality/mor- 
bidity, especially in heart failure. A BNP value at a 
certain time point is closely associated with the 
prognosis in terms of mortality and morbidity. In 



addition, heart failure therapy guided by the 
change in BNP has been shown to significantly 
reduce the number of cardiovascular events com- 
pared with treatment according to guideline rec- 
ommendations.*" The introduction of a reliable 
surrogate for mortality would certainly be most 
welcome. How^ever, the demands on such a surro- 
gate will obviously have to be substantial. It must 
be ascertained that changes in the surrogate end- 
point are really closely associated with changes in 
mortahty and/or morbidity. Regarding BNP, there 
are many questions to be answered before it can be 
widely accepted as a surrogate for mortaUty/mor- 
bidity. For example, httle is known about whether 
the reactions of BNP to certain interventions (eg, 
0-blocker therapy) are parallel to the effect of 
those interventions on mortality and morbidity. 
The value of BNP may also differ depending on the 
group of patients, the time points of the disease 
process, concomitant therapy, and concomitant 
diseases. Those and other questions must be ade- 
quately answered before BNP or any surrogate for 
mortality and morbidity can be accepted. 

Real Life Efficacy Versus Clinical 
Trial Efficacy 

As stated in the introduction, the ultimate pur- 
pose of a clinical trial is that the findings should be 
applicable to the patients in our ever>day clinical 
practice. As mentioned previously, patients par- 
ticipating in clinical trials may be selected in var- 
ious aspects that are not apparent. Thus, although 
the study population may resemble the real-life 
patients with regard to age, gender, concomitant 
diseases, and therapy, and everything else you can 
put your finger on, it may be questionable to ex- 
trapolate the trials result into cHnical practice. A 
factor strongly contributing to this is the different 
behavior of patients in clinical trials and in real 
life. 

Almost 100% of the patients in cUnical trials 
take their medicine exactly as prescribed, whereas 
it is well known that real-hfe patients often do not. 
It has been shown that more than one third of 
patients either do not take 1 or more prescribed 
drugs at all or take incorrect doses of at least 1 
drug,^^ This problem increases with age and num- 
ber of prescribed drugs. Therefore, even if the 
drug is proven efficacious in a clinical trial, pa- 
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tients must take it for it to work in real life. Obvi- 
ously, the side effects of a drug are itnportaat 
regarding patient compliance and» consequently, 
to the efficacy of the drug. This can be illustrated 
by an example. Drug A and B both are significantly 
better than placebo. Drvig A is the established 
therapy, whereas drug B is the new one. The 2 
drugs are compared in a clinical trial. There is no 
significant difference in morbidity and survival 
between the 2 drugs, but the confidence intervals 
are more on the negative side of drug B. However, 
drug B is substantially better tolerated. The rec- 
ommendation after the trial is, of course, to keep 
drug A as firsi-hne treatment and to use drug B in 
patients who cannot tolerate drug A. The interest- 
ing question is what the real-Ufe benefit of the 
respective drug would be. If drug B was to replace 
drug A, it might do more good in terms of mortal- 
ity and morbidity than drug A in real life, if con- 
siderably more patients would take drug B. 

The ELITE II trial might be considered applica- 
ble to the above example. The effects on mortal- 
ity and morbidity of the angiotensin receptor 
blocker (ARB) losartan was not significantly dif- 
ferent from that of the ACE inhibitor captopril 
during L5 years of follow-up in 3,152 patients 
with heart failure. The mortality odds ratio for 
losartan versus captopril was 1.13, with a 95% 
confidence inter\^ai of 0.95 to 1.35 (P = .16). 
However, approximately 30% more patients with- 
drew prematurely because of adverse events in the 
captopril group than in the losartan group (14.5% 
v9.4%,P < .001). 

If losartan was to replace ACE inhibitors in clin- 
ical practice, could that possibly be beneficial to 
the general population of patients with heart 
failure? To answer that question, we also have to 
consider the physician's behavior. As previously 
mentioned, many physicians avoid prescribing 
ACE inhibitors because of a fear of side effects. 
This might lo some extent explain the relatively 
low use of ACE inhibitors in the heart failure pop- 
ulation, shown to be around one third of all pa- 
tients with heart failure in western society."^^*^^ If 
physicians were to prescribe ARBs more than they 
currently prescribe ACE inhibitors, and if patients 
were to be more compliant to ARBs than to ACE 
inhibitors, ARBs could actually do more good 
than ACE inhibitors in real life. We do not know if 
the above example is valid* but it is certainly 



worth considering the real-life effects compared 
with the clinical trial effects when evaluating the 
efficacy of a treatment regimen. 

Publication Bias 

What you read is what you know. Only trials that 
have been published have the ability to affect how 
guidelines are written and how physicians prescribe 
drugs. However, published trials do not provide the 
complete truth regarding the findings of clinical 
studies. It is well known that there is substantial 
publication bias. A positive trial stands a better 
chance of being accepted for publication than a trial 
showing no difference between 2 treatment regi- 
mens, or a trial showing that the new agent is even 
less efficacious than the established one. Further- 
niore, many negative or neutral trials never even 
reach the submitted manuscript stage. An example 
of the latter is that most mortality and morbidity 
trials on positive inotropic agents in heart failure 
have never been published because of a negative 
effect compared with placebo. These trials were suf- 
ficiently large and conclusive to merit publication in 
a major journal but were never submitted. 

It is just as important to know about negative or 
neutral trials as it is of positive ones. This percep- 
tion has during the past years been gaining 
ground, and measures have been taken to secure 
publication of all well-done trials, irrespective of 
the outcome. An example of such measures is the 
launch of journals with the policy to do just that, 
such as Current Controlled Trials in Cardiovascu- 
lar Medicine. Another example is BioMed Central 
on the internet, with the aim to make primary 
biomedical research freely available to all. The 
purpose is to officially register all research efforts 
as they start, which will secure that no clinical trial 
disappears on the way. Thus, this kind of journal 
policy may also affect publication bias owing to 
withdrawn sponsor support or unwillitigness of 
the sponsor to publish negative trial results. In 
addition, it is important that researchers in the 
study protocol clearly state the aim to publish the 
results of the trials irrespective of outcome. 

Conclusion 

Clinical trials constitute the backbone of evi- 
dence-based medicine. Indeed, clinical trials in 
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various cardiovascular conditions have supplied 
us wilh exlremely valuable information during 
the last 20 years. Without these trials, prognosis in 
hypertension, myocardial infarction, and heart 
failure most likely would have been worse than it 
is today. Yet, the design of most trials has not been 
optimal, and knowing what we know today, we 
probably would have designed most of them dif- 
ferently. The design of clinical trials in cardiovas- 
cular medicine has improved during the last years. 
To secure further improvements, it is important 
that we continuously discuss the pitfalls and prob- 
lems outhned in this article. There is also consid- 
erable room for improvement regarding the inter- 
pretation and implementation of the trial results. 
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